Covering Number for Efficient Heuristic-based POMDP Planning
نویسندگان
چکیده
The difficulty of POMDP planning depends on the size of the search space involved. Heuristics are often used to reduce the search space size and improve computational efficiency; however, there are few theoretical bounds on their effectiveness. In this paper, we use the covering number to characterize the size of the search space reachable under heuristics and connect the complexity of POMDP planning to the effectiveness of heuristics. With insights from the theoretical analysis, we have developed a practical POMDP algorithm, Packing-Guided Value Iteration (PGVI). Empirically, PGVI is competitive with the state-of-the-art point-based POMDP algorithms on 65 small benchmark problems and outperforms them on 4 larger problems.
منابع مشابه
Covering Number: Analyses for Approximate Continuous-state POMDP Planning (Extended Abstract)
To date, many theoretical results on discrete POMDPs have not yet been extended to continuous-state POMDPs, due to the infinite dimensionality of the belief space in a continuousstate case. In this paper, we define a distance in the `nmetric space with respect to a partitioning representation of the continuous-state space, and formalize the size of the search space reachable under inadmissible ...
متن کاملCovering Number as a Complexity Measure for POMDP Planning and Learning
Finding a meaningful way of characterizing the difficulty of partially observable Markov decision processes (POMDPs) is a core theoretical problem in POMDP research. State-space size is often used as a proxy for POMDP difficulty, but it is a weak metric at best. Existing work has shown that the covering number for the reachable belief space, which is a set of belief points that are reachable fr...
متن کاملDistribution over Beliefs for Memory Bounded Dec-POMDP Planning
We propose a new point-based method for approximate planning in Dec-POMDP which outperforms the state-of-the-art approaches in terms of solution quality. It uses a heuristic estimation of the prior probability of beliefs to choose a bounded number of policy trees: this choice is formulated as a combinatorial optimisation problem minimising the error induced by pruning.
متن کاملWhat makes some POMDP problems easy to approximate?
Point-based algorithms have been surprisingly successful in computing approximately optimal solutions for partially observable Markov decision processes (POMDPs) in high dimensional belief spaces. In this work, we seek to understand the belief-space properties that allow some POMDP problems to be approximated efficiently and thus help to explain the point-based algorithms’ success often observe...
متن کامل